Skip to content

Move test repetition down to the test case level#1528

Open
harlanhaskins wants to merge 4 commits intomainfrom
harlan/getting-those-reps-in
Open

Move test repetition down to the test case level#1528
harlanhaskins wants to merge 4 commits intomainfrom
harlan/getting-those-reps-in

Conversation

@harlanhaskins
Copy link
Copy Markdown
Contributor

@harlanhaskins harlanhaskins commented Feb 5, 2026

This changes the runner such that "repeat until success" or the other repetition policies repeat only the affected test case, rather than repeating the entire test plan.

Fixes #1392 and rdar://130508488

Motivation:

The previous behavior was unexpected and does not match the behavior of XCTest. This changes the behavior to align the behaviors (in the way that users expect).

Modifications:

  • Adds a configuration option requiring clients to opt in to this behavior (for now, until we change the default).
  • Updates the Runner to move the iteration into a helper function
  • Adds this helper function within _runTestCase (if opted in)
  • Replaces the plan-level iteration with the helper function (if opted out)
  • Mirrors the plan-level iteration tests to test-case level iteration tests that test the same conditions with the new behavior

Checklist:

  • Code and documentation should follow the style of the Style Guide.
  • If public symbols are renamed or modified, DocC references should be updated.

@harlanhaskins
Copy link
Copy Markdown
Contributor Author

Converting to a draft for now; there's follow-on discussions we need to have about behavioral changes before we can actually sink this repetition down

@harlanhaskins harlanhaskins force-pushed the harlan/getting-those-reps-in branch from 03ba197 to 8b09896 Compare February 24, 2026 22:18
@harlanhaskins harlanhaskins force-pushed the harlan/getting-those-reps-in branch from 8b09896 to 2f6657b Compare March 9, 2026 22:52
@harlanhaskins harlanhaskins marked this pull request as ready for review March 24, 2026 19:31
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Support retry-on-failure for only the failing @Test cases

1 participant